2-D Spatial Smoothing for Multipath Coherent Signal Separation

0018-9251/98/$10.00 ID 1998 IEEE As the digital signal processing technologies advance, the use of adaptive arrays to combat multipath fading and to reduce interference becomes increasingly valuable as a means of adding capacity to mobile communications. There are many optimum adaptive array combining algorithms. Among them, the high resolution direction finding based constrained adaptive beamforming [l-31 and the reference signal based optimum spatial diversity array combining [4, 51 are considered most applicable in time division multiple access (TDMA) wireless communications and have attracted the most attention. In this work, we only consider the constrained adaptive beamforming approach. The key to making this technique effective in multipath fading environment is accurate direction of arrival (DOA) estimation of coherent or highly correlated signals. The optimal weighted subspace fitting (WSF) DOA estimation [2] algorithm has been previously proposed to identify the coherent signals in a mobile communication environment. But the WSF and other optimal DOA estimation algorithms such as deterministic maximum likelihood (DML) [6] require optimization of a multidimensional nonlinear criterion function. Since this may be a difficult and time-consuming task, less expensive suboptimal approaches such as the multiple signal classification (MUSIC) algorithm [7] and estimation of signal parameters via rotational invariance techniques (ESPRIT) [8] are often employed. The MUSIC algorithm involves one-dimensional search, compared with multidimensional search inherent in the optimal approaches. However, computational efficient suboptimal DOA estimation techniques have the drawback of severe degradation of the estimation accuracy [ 121 in the presence of highly correlated or coherent signals. (Coherent signals also cause signal cancellation [ 131 in adaptive beamforming algorithms such as minimum-variance distortionless response (MVDR) and linearly constrained minimum-variance beamforming (LCMV) algorithms [9-111.) The multidimensional methods were still conceived to be more appropriate than the suboptimal approaches for coherent interference environment, despite their computational requirements.